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Abstract 

Multi-objective optimisation is regarded as one of the most promising ways for dealing with constrained optimisation problems 
in evolutionary optimisation. This paper presents a theoretical investigation of a multi-objective optimisation evolutionary algorithm 
for solving the 0-1 knapsack problem. Two initialisation methods are considered in the algorithm: local search initialisation and 
greedy search initialisation. Then the solution quality of the algorithm is analysed in terms of the approximation ratio. 


I. Introduction 

Consider the problem of maximizing an objective function, 


max/(:?), subject to g{x) < 0. (1) 

X 

The above constrained optimisation problem can be transferred into an unconstrained bi-objective optimisation problem. That 
is to optimize the original objective function plus to minimize the constraint violation simultaneously: 


r max£/(f), 
\ min£z;(5'), 


where v{x) is the degree of constraint violation, given by 


f 0, if g{x) < 0, 
[ g{x), otherwsie. 


( 2 ) 

( 3 ) 


The use of multi-objectives for single-objective optimisation problems could be traced back to 1990s m. This methodology 
has been termed multiobjectivisation El. Using multiobjectivization sometimes may help the search more efficient as shown 

in El-El. 

According to the survey El. multi-objective optimisation is regarded as one of the most promising ways for dealing with 
constrained optimisation problems in evolutionary optimisation. A constrained optimisation problem is often transformed into 
a bi-objective optimisation problem, in which the first objective is the original objective function and the second objective is 
the degree of constraint violation El-EIl. After this transformation, Pareto dominance is frequently employed to compare 
individuals. Currently the research in this area is very active El. For example, a self-adaptive selection method is proposed 
recently in lfT2l . which aims to exploit both non-dominated solutions with low constraint violations and feasible solutions with 
low objective function values. Multi-objective optimisation is combined with differential evolution in ifTSl and an infeasible 
solution replacement mechanism is proposed. A dynamic hybrid framework is presented in lfT4l . where the global and local 
search models are implemented dynamically according to the feasibility proportion of the population. 

This paper aims at analysing the solution quality of evolutionary algorithms (EAs) in terms of the approximation ratio. It 
is not intended to demonstrate that EAs are able to compete with problem-specific approximation algorithms, since this is 
unlikely in most cases. Nevertheless, it is still necessary and important to understand the solution quality of EAs, so the EAs 
with arbitrarily bad solution quality could be avoided in applications. The analysis of the approximation performance of EAs 
has attracted a lot of interests in recent years ES-iini. 

This paper investigate an existing multiobjective optimization-based EA El (MOEA) for solving constrained optimisation 
problems. The MOEA originally is designed for continuous optimization. Here it is adapted for solving the 0-1 knapsack 
problem. Although experiment results show its performance is good, no theoretical analysis exists for this MOEA E). This 
motivates our rigorous analysis. 

The remainder of the paper is organized as follows. The 0-1 knapsack problem and approximation ratio are introduced in 
Section |II] The MOEA with the local search initialisation is analysed in Section |III] . Section |IV] is devoted to the analysis of 
the MOEA with the greedy search initialisation. Section IV] concludes the article. 
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II. 0-1 Knapsack Problem and Approximation Ratio of Solutions 


Given an instance of the 0-1 knapsack problem with a set of weights Wi, values Vi, and capacity kP of a knapsack, the task 
is to find a binary string afmax so as to maximize the objective function, 

n n 

max/(af) = ViXi, subject to WiXi < W, (4) 

2=1 2=1 


where x = {xi • • • x„) is a binary string. Xj = 1 if item i is selected in the knapsack; otherwise Xi = 0. 

A feasible solution is a knapsack represented by an x which satisfies the constraint, that is ^ infeasible 

one is an x that violates the constraint. The string (0... 0) represents a null knapsack. Without loss of generality, assume that 
a feasible solution always exists and n is large. 

There exist well-known approximation algorithms for the 0-1 knapsack problem ITSl . ITOl . Probably the simplest one is 
the greedy search m whose worst-case approximation performance ratio equals to 1/2 and time complexity is 0{n) plus 
0(n log n) for the initial sorting. A polynomial-time approximation scheme has been introduced in ifTSll whose worse-case 
performance is k/ (1 + k) given an integer parameter k and its time complexity is Furthermore, a fully-polynomial- 

time approximation scheme is well-known m whose time complexity is 0{n/f^') plus 0(n log n) for the initial sorting given 
a parameter e > 0. 

In evolutionary optimisation, the 0-1 knapsack problem has been taken as a benchmark in computer experiments ll20l . ||2T]| 
for evaluating the performance of various constraint-handling techniques. It is also one of favourite problems used in the 
theoretical study of EAs ll22l . Il2^ . 

In order to assess the solution quality of an EA, an evolutionary approximation algorithm is defined as below. It follows the 
definition of conventional a-approximation algorithms Il24l Definition 1]. 

Definition 1: An EA is an a-approximation algorithm for a constrained optimisation problem if for all instances of the 
problem, the EA can produce a feasible solution in polynomial running time, whose objective function value is within a factor 
of a of that of an optimal solution. The running time of an EA is the expected number of function evaluations. 

In a maximisation problem (assume /(Xmax) > 0), a feasible solution x is called to have an a-approximation ratio if it 
satisfies 


fix) 


> a. 


(5) 


In order to prove that an EA is not an a-approximation algorithm, it is sufficient to show that an EA needs exponential 
running time to obtain a feasible solution with an a-approximation ratio in one instance of the problem. 


III. Analysis of MOEA with Local Search Initialisation 


The 0-1 knapsack problem can be transformed into a bi-objective optimisation problem, that is to maximize the objective 
function /(x) and to minimize the constrain violation u(x), where i;(x) is defined by 


v{x) 


0; if X is feasible, 

Yfli=i Otherwise. 


( 6 ) 


Although a constrained optimisation problems can be converted into a bi-objective optimisation problem, there exists an 
essential difference between it and general multi-objective optimisation problems Q- The target of general multi-objective 
optimisation is to obtain a final population with a diverse non-dominated individuals uniformly distributed on the Pareto front. 
However in the bi-objective optimisation problem derived from contained optimisation, the target is to obtain the optimal 
feasible solution of the original constrained optimisation problem. Consequently, there is no need to care about the uniform 
distribution of the resulting solutions on the Pareto front. 

The MOEA adopted in this section is a variant of an existing MOEA proposed in ||9|- The fundamental idea in this MOEA is 
that non-dominated individuals in a children population are chosen and replace dominated individuals of the parent population. 
The algorithm, based on Model 1 of ii, is described in Algorithm [T] for solving the 0-1 knapsack problem. 

It should be pointed out that the running time of EAs is dependent on initialisation. Two initialisation methods are considered 
in the paper: initialisation by the local search and by the greedy search. In this section, we investigate the first one: the local 
search initialisation, described in Algorithm ID This initialisation does not only produce both feasible solutions (local optima), 
but also infeasible solutions. Bitwise mutation flips each bit of a binary string with probability T. Population size N is set to 
a large constant and for the sake of analysis, assume that A^/4 is an integer. 

We show that the solution quality of the MOEA with the local search initialisation might be arbitrarily bad using the 
following instance of the 0-1 knapsack problem. 

Instance 1: In the following table, H, I and J represent index sets. Eor the sake of simplicity, assume that ^ and ^ are 
integers. Eixing a constant a € (0,1), choose n an enough large integer so that n > ^ and ^ > 


2 
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Algorithm 1 MOEA m _ 

1: initialize population $o; 

2 : for t = 0,1,2, • ■ • do 

3: perform bitwise mutation and generate a children population ^t.a with N individuals; 

4: evaluate the values of /(x) and u(x); 

5: choose the non-dominated individuals from population ^ and assume there are k non-dominated individuals, denoted 

as {fi, • • • ,Xk}; 

6: set an intermediate population ^t.b ^ ‘f*; 

7: for I = 1, • • • , fc do 

8: let m be the number of individuals in ^t.b which are dominated by Xi, 

9: if TO = 0 then 

10: do nothing; 

11: else if TO = 1 then 

12: the corresponding dominated individual is replaced by Xi, 

13: else 

14: if the dominated individuals are feasible then 

15: the individual with the smallest objective function value is replaced by xu 

16: else 

17: one of the dominated individuals is randomly chosen and replaced by Xi, 

18: end if 

19: end if 

20 : end for 

21: set the next generation population 

22 : end for 


Algorithm 2 Local Search Initialisation 
1 : set X = (0 ■ • • 0); 

2 : while X is feasible do 

3: flip one 0-valued bit of x into 1-valued, denote it by y-, 

4: if y is feasible then 

5: let X ■<— y; 

6 : else 

7: let X ^ y with probability 1/2; 

8 : end if 

9: end while 

10: repeat the above steps until N individuals are produced. 


Let Xmax represent the global optimum such that xi = 1 and other bits Xi = 0. The global optimum is unique. Its objective 
function value is 


f (^^max) — 7Z. 


( 7 ) 


Let xioc represent a 
value is 


local optimuirQ such that 


^ bits Xi = 1 (where i € I) and other bits Xi = 0. Its objective function 


/(xioc) = (8) 

fixioc) is the second largest objective function value among feasible solutions. The number of such local optima xioc is 
experiential in n, 



Let Xvioi denote an infeasible solution such that Xi = 1 for ^ indexes i G I, one i G J, and Xi = 0 for other i. Its objective 
function value and violation value are 


fixvioi) = fixioc) + a''"", i'(^vioi) = 


( 10 ) 


*A feasible solution x is called a local optimum if f(y) < f{x) for any feasible solution y within Hamming distance d{x,y) = 1. 
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The number of such infeasible solutions Xvioi is exponential in n, 


t V-1 ^ n-l 

an 


( 11 ) 


Let Xvio 2 denote an infeasible solution such that Xi = 1, Xi = 1 for one i G J, and Xi = 0 for any other i. Its objective 
function value and violation value are 

f{Xvio2) = /(Xmax) + a''"", v{Xyio2) = . (12) 

It is easy to verify that the degree of constraint violation u(xvioi) and u(xvio 2 ) (= is the minimum among all infeasible 

solutions. 

Instance 1 is hard since Hamming distance between a local optimum xioc and the unique global optimum Xmax is large 
> an/2 and the the number of local optima is exponential in n. Since the selection used in the MOEA is that non-dominated 
individuals in a children population are chosen and replace dominated individuals of the parent population, it prevents individuals 
moving from the 2nd best fitness level to the best fitness level. Thus the EA needs exponential time to leave the absorbing 
basin of the local optima. 

Theorem 1: Eor Instance 1 and any constant a G (0,1), the MOEA with the local search initialisation needs H(n'^) running 
time to find an a-approximation solution in the worst case. 

Proof: After the initialisation, individuals generated by the local search may include Xmax, local optima xioc and infeasible 
solutions X with Hamming distance H{x, Xmax) = 1 or H{x, xioc) = 1. The worst case is that after initialisation, population $0 
is composed of N local optima xioc and infeasible solutions Xviot ■ Notice that the number of local optima xioc and infeasible 
solutions Xvioi is exponential in n. The individuals in $0 may be chosen to be different. 

Assume that in the tth generation, population is composed of N local optima xioc and infeasible solutions Xviot. The 
approximation ratio between /(xioc) and /(xmax) is 

fixioc)_ _ a 
" “ 2 


= — < a. 


(13) 


/ (Xmax) 

Since a could be any constant, the above approximation ratio could arbitrarily bad. 

As we know that Xmax is the unique solution satisfying /(xmax) > /(^loc), it is sufficient to prove that the EA needs 
exponential running time to generate Xmax- 

Eirst we consider the event of mutating xioc or Xvioi into a child y. The event can be decomposed into the following mutually 
exclusive and exhaustive sub-events. 

1) y is a feasible solution such that f{y) < /(xioc). 

Obviously y will not dominate xioc. At the same time, y will not dominate Xvioi since f{y) < /(xioc) < /(^vioi)- Thus 
y will not dominate any individuals in <I>t and cannot be selected into the next generation population, 
y is a feasible solution such that f{y) = /(xioc), that is, y is a xioc- 
y is a feasible solution such that f(y) > f{x\oc), that is, y is the global optimum Xmax- 

In this case, all 1-valued bits x^ (where i G I) must be flipped from 1 to 0. The probability of the event happening is at 
most 


2 ) 

3) 


o\- 

. n 


(14) 


4) y is an infeasible solution such that v{jj) > x{xvioi). 

In this case, y will not dominate xioc and Xvior, so it cannot be selected into the next generation population. 

5) y is an infeasible solution such that v{y} = v{xvioi), that implies, 

a) either y is Xvioi; 

b) or y is an infeasible solution Xvio 2 - 

In the second case, all 1-valued bits Xi (where i G I) must be flipped from 1 to 0. The probability of the event 
happening is 


O 


( 15 ) 
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From the above analysis, we observe that only when either Xmax or an infeasible solution Xvio 2 is generated via mutation, the 
population status could be changed. Otherwise the population is still composed of N solutions xioc and Xvioi- The probability 
of the former event happening is O (T) ^ . 

Next we analyse the role of using a population. Consider the event that a population includes either Xmax or an infeasible 
solution Xvio 2 is generated via mutation. Since N parents are selected and mutated independently, the probability of the event 
happening is NO . Thus the expected number of generations for the EA to reach x^ax is • Since there are 

N fitness evaluations at each generation, the expected number of fitness evaluations is 17 (n'^) . The required conclusion is 
then proven. ■ 


IV. Analysis of MOEA with Greedy Search Initialisation 

In order to produce a good quality solution with a guaranteed approximation ratio, a natural idea is to combine an EA 
with an approximation algorithm: first we apply an approximation algorithm to producing approximation solutions as the 
initial population, and then apply the MOEA to searching a global optimum. In this section, we consider a i-approximation 
algorithm to implement the initialisation. It a variant of the greedy search ifTSl Section 2.4], described in Algorithm [3 Notice 
that the initialisation does not only produce feasible solutions (local optima), but also infeasible solutions. 


Algorithm 3 Greedy Search Initialisation 
1 : sort all the items via their values so that pi > ■ ■ ■ > Pn', 

2: then greedily add the items in the above order to the knapsack as long as adding an item to the knapsack does not exceeding 
the capacity of the knapsack. Denote the solution by Xa', 

3: resort all the items via the ratio of their values to their corresponding weights so that 

4: Then greedily add the items in the above order to the knapsack as long as adding an item to the knapsack does not 
exceeding the capacity of the knapsack. Denote the solution by Xf,; 

5: put Xa and Xb into the initial population; 

6: repeat the above procedure until individuals are produced; 

7: for each of these ^ individuals, add one item and then ^ infeasible solutions are produced. 


Using the greedy search initialisation, we may find the global optimal solution of Instance 1 during the initialisation phase. 
Eurthermore, since the greedy search is a 1/2-approximation algorithm for the 0-1 knapsack problem, the MOEA with the 
greedy search initialisation is an evolutionary 1/2-approximation algorithm too. The advantage of using an EA is the ability 
to obtain the global optimum due to the use of bitwise mutation. 

In the following we answer the question: can the MOEA with the greedy search initialisation find a solution with the 
approximation ratio better than 1/2? Through analysing the instance described below, we obtain a negative answer. 

Instance 2: In the following table, H, I, J and K represent index sets. Eor the sake of simplicity, assume that n/4 is an 
integer. 



H 

1 

J 
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o Tt 

t 4 

11 _i -| ft 
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Vi 
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n 
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1 
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Let x„iax represent the unique global optimum such that x^ = 1 for any i € H and x^ = 0 for any other i. Its objective 
function value is 


/(Xmax) = 2n. (16) 

Let xioc represent a local optimum such that Xi = 1 for one z G /, any z G J and j —2 indexes z G AT; Xj = 0 for all other 
z. Its objective function value is 

/(fioc)=n + 2+(|-2)n-3. (17) 

/(xioc) is the second largest objective function value among feasible solutions. 

Let Xvioi represent an infeasible solution such that Xi = 1 for one z G /, any z G J and j — I indexes z G AT; Xi = 0 for 
all other z. Its objective function value and violation value are 

/(Xviol) = /(xioc) + 7Z ) 'u(Xviol) = 'T7Z j 

4(1 + n 


( 18 ) 
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Let Xvio 2 represent an infeasible solution such that Xi = 1 for any i ^ H and one i G K, and Xi 
objective function value and violation value satisfy 

y*(Xvio 2 ) J^(^niax) “b , r’(^vio 2 ) ~77^ I 4 T' 

4(1 + n~^) 


0 for any other i. Its 


(19) 


Let Xvio 3 represent an infeasible solution such that xt = 1 for any i G H and at least one i G J, and Xi = 0 for any other 
i. Its objective function value and violation value satisfy 

Tl~‘^ Tl~^ 

/(^max) ^ /(^vios) — /(^max) “b ^ 7 9 ^ ^^(Xvios) ^ ^ ■ ( 20 ) 

Instance 2 is a hard problem to EAs using bitwise mutation since Hamming distance between a local optimum xioc and 
the unique global optimum Xmax is large. Another trouble is the number of local optima which is exponential in n. Thus it is 
difficult for a population to leave the absorbing basin of the local optima. 

Theorem 2: For Instance 2, the MOEA with the greedy search initialisation can find a ^-approximation solution after 
initialisation. But in the worst case, it needs 0(n?) running time to find a -b ^ + 2 ^)-approximation solution. 

Proof: The first conclusion is trivial due to the use of the greedy search. After the initialisation, individuals generated by 
the greedy search are local optima xioc and infeasible solutions x with Hamming distance iJ(x, xioc) = 1- The local optimum 
xioc has an approximation ratio given by 

fjxioc) ^ W + 2 + (§ - 2) /I 11 J_\ ,21) 

/(^^max) 2n V2’2 n 2n^ J ' 

The proof of the second conclusion is similar to that of Theorem 1. The worst case is that after initialisation, population $o 
is composed of N local optima xioc and infeasible solutions Xvioi- Notice that the number of local optima xioc and infeasible 
solutions Xvioi is exponential in n. The individuals in $o could be chosen to be different. 

Assume that in the fth generation, population is composed of N different local optima xioc and infeasible solutions Xvioi- 
Let y be a child mutated from a parent x. The event can be decomposed into the following mutually exclusive and exhaustive 
sub-events. 

1 ) y is feasible such that /(y) < /(xioc). 

According to the selection based on the Pareto-dominance, y will not be selected to the next generation population. 

2 ) y is feasible such that /(y) = /(xioc), that is, y is a local optimum xioc- 

3) y is feasible such that /(y) > /(xioc), that is, y is Xmax- 

In this case, any 1-vaIued bit Xi where i G IU J U K must be flipped from 1 to 0, and xi and X 2 must be flipped from 
0 to 1. Thus at least j bits must be flipped. The probability of this event happening is at most 0{n~i). 

4) y is infeasible such that u(x) = , that is 

• either y is Xvioii 
. or y is Xvio 2 - 

In the second case, except one 1-valued bit Xi where i G K, any other 1-valued bit Xi where i G IL} J U K must be 

flipped from 1 to 0, and xi and X 2 must be flipped from 0 to 1. Thus at least j bits must be flipped. The probability 

of this event happening is at most 0(n~^). 

5) y is infeasible such that v(x) < that means, y is Xvio 3 - 

In this case, any 1-valued bit Xi where i G I U K must be flipped from 1 to 0, and xi and X 2 must be flipped from 0 
to 1. Thus at least j bits must be flipped. The probability of this event happening is at most 0{n~T). 

6 ) y is infeasible such that u(x) > ^ "'tH not be selected since it is dominated by Xvioi- 

From the above analysis, we observe that the probability of generating a non-xioc and non-Xvioi child and selecting it to 
the next generation population is small, that is 0(n“7). 

Next we analyse the role of using a population. Consider the event that the next generation population includes a non-xioc 
or non-Xvioi child. Since N parents are mutated independently, the probability of the event happening is at most NO{n~^). 
This implies that the expected number of generations for the EA to reach Xmax is at least ). Since there are N fitness 

evaluations at each generation, the expected number of fitness evaluations is The required conclusion is proven. ■ 

The above theorem shows that the MOEA with the greedy search initialisation finds a ( 1 +n + 2^) -approximation solution in 
) running time. As n ^ -l-oo, the approximation ratio goes towards 1/2. In other words, the MOEA doesn’t substantially 
improve the solution quality since the greedy search already produces a 1/2 approximation solution during initialisation. 


V. Conclusions 

This paper has assessed the solution quality of an existing MOEA Q for solving the 0-1 knapsack problem. The solution 
quality of an EA is measured in terms of the approximation ratio. Two different initialization methods are analysed in the 
MOEA: local search initialisation and greedy search initialisation. 
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When the initial population is produced by the local search, the solution quality of the MOEA might be arbitrarily bad in 
some instance. That is, given any constant a G (0,1), the MOEA needs O(n^) running time to find an a-approximation 
solution in the worst case in some instance. 

When the initial population is produced by the greedy search, the MOEA may guarantee a 1/2-approximation solution within 
polynomial time. However, this improvement is caused by the use of the greedy search, rather than the MOEA itself. In some 
instance, the MOEA with the greedy search initialisation needs O(n^) running time to find a (5 + ^ + 5 ^)"Approximation 
solution. In other words, the MOEA doesn’t substantially improve the solution quality comparing with the greedy search. 

Other types of initialisation, such as random initialisation, are not considered in the current paper. It is left for future work. 
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